1. Field of the Invention
The present invention relates to image processing apparatuses and methods, recording media, and programs, and more particularly relates to an image processing apparatus and method suitable for use in generating a wide dynamic range color image including a plurality of pixels having wide dynamic range color components on the basis of a color-and-sensitivity mosaic image including pixels, each pixel having a color component differing from that of a neighboring pixel and being captured with a sensitivity differing from that with which the neighboring pixel is captured, to a recording medium, and to a program.
2. Description of the Related Art
Solid-state image devices, such as CCDs (Charge Coupled Devices) and CMOS's (Complementary Metal-Oxide Semiconductors), are widely used in image capturing apparatuses, such as video cameras and digital still cameras, component inspectors in the field of factory automation (FA), and optical measuring instruments, such as electronic endoscopes in the field of medical electronics (ME).
There are known techniques that combine optical intensity signals generated by measurement with different sensitivities at individual pixels, thereby improving the dynamic range of an image capturing apparatus using a solid-state image device or the like. Five such known techniques will now be described.
A first known technique is a method of optically measuring incident light rays split into a plurality of optical axes having different transmittances by solid-state image devices disposed on the individual optical axes (e.g., see Japanese Unexamined Patent Application Publication No. 8-223491). According to the first known technique, a plurality of solid-state image devices and a complicated optical system for splitting light rays are necessary. This is disadvantageous in terms of cost and size reduction.
A second known technique is a method of capturing, by a single solid-state image device, a plurality of images by splitting exposure time into a plurality of periods and then combining the captured images (e.g., see Japanese Unexamined Patent Application Publication No. 8-331461). According to the second known technique, pieces of information generated by measurement with different sensitivities are those captured at different times in which the corresponding exposure time periods are of different lengths. A dynamic scene in which optical intensity changes from time to time cannot be captured accurately.
A third known technique is a method of capturing an image by a single solid-state image device in which each pair of adjacent photo sensors on an image capturing side of the solid-state image device is associated with one pixel of an output image, and the paired adjacent photo sensors are set to different sensitivities (e.g., see Japanese Unexamined Patent Application Publication No. 59-217358). The sensitivities of photo sensors of the solid-state image device may be changed by covering the individual photo sensors with filters having different transmittances.
The third known technique is advantageous over the first known technique for cost and space reduction. The third known technique is advantageous over the second known technique in that the former can accurately capture a dynamic scene. According to the third known technique, each pair of adjacent photo sensors is associated with one pixel of the output image. To ensure the resolution of the output image, an image device including photo sensors, the number of which is several times the number of pixels of the output image, is necessary. The unit cell size is thus increased.
A fourth known technique is a method of generating a wide dynamic range image signal by applying a mechanism in which each photo sensor associated with one pixel of an output image has a different exposure to a normal dynamic range image device and performing predetermined image processing on a generated image signal. The mechanism in which each photo sensor has a different exposure is achieved by generating a spatial sensitivity pattern by changing the light transmittance and/or aperture of each photo sensor (e.g., see S. K. Nayar and T. Mitsunaga, “High Dynamic Range Imaging: Spatially Varying Pixel Exposures”, Proc. of Computer Vision and Pattern Recognition 2000, Vol. 1, pp. 472-479, June, 2000).
According to the fourth known technique, each photo sensor has one type of sensitivity. Although each pixel of a captured image has information with a dynamic range intrinsic in the image device, a wide dynamic range image can be generated by performing predetermined image processing on a generated image signal so that all pixels have a uniform sensitivity. Since all photo sensors are exposed at the same time, a dynamic subject can be captured accurately. Since each photo sensor is associated with one pixel of the output image, the unit cell size is not increased.
The fourth known technique is based on the assumption that a monochrome image is to be generated. A technique for generating a color image is not established yet.
A fifth known technique is a method of capturing a wide dynamic range color image signal by applying a mechanism in which each photo sensor associated with one pixel of an output image has a different exposure and outputs a color component differing from that of an adjacent photo sensor to a normal dynamic range image device and performing predetermined image processing on a generated color-and-sensitivity mosaic image. The mechanism in which each photo sensor outputs a color component differing from that of an adjacent photo sensor is achieved by covering the individual photo sensors with color filters (e.g., see Japanese Unexamined Patent Application Publication No. 2002-209223).
According to the fifth known technique, a wide dynamic range color image can be generated by performing predetermined image processing on a captured color-and-sensitivity mosaic image. In the course of image processing, computations must be done to generate a brightness image in which each pixel has a brightness signal and a color difference image in which each pixel has a color difference signal. A memory for storing these images is thus necessary. There is a demand for reducing the computations and circuit size.